Shape recovery from multiple views: a parallax based approach

Given two arbitrary views of a scene under central projection, if the motion of points on a parametric surface is compensated, the residual parallax displacement eld on the reference image is an epipolar eld. The parallax magnitude at a point, after suitable scaling, is an a ne invariant; if the surface aligned is a plane, it is directly proportional to the height of the point from the plane and inversely proportional to its depth from the camera. We exploit the above result to infer 3D height information from oblique aerial 2D images. We use direct methods to register the aerial images, develop methods to infer height information under the following three conditions: (i) focal length and image center are both known, (ii) only the focal length is known, and (iii) both are unknown. We use the invariance property of the scaled parallax magnitudes to combine multiple frame information to obtain accurate heights, and to extrapolate new views from a given set of views (i.e., in photogrammetric terms, to achieve \transfer"). We use the view extrapolation process to construct a panoramic mosaic image by combining multiple views that is accurate in terms of 3D positions of surfaces.